The Margin Vector, Admissible Loss and Multi-class Margin-based Classifiers

We propose a new framework to construct the margin-based classifiers, in which the binary and multicategory classification problems are solved by the same principle; namely, margin-based classification via regularized empirical risk minimization. To build the framework, we propose the margin vector which is the multi-class generalization of the margin, then we further generalize the concept of admissible loss in binary classification to the multi-class cases. A multi-class margin-based classifier is produced by minimizing the empirical margin-vector-based admissible loss with proper regularization. We characterize a class of convex losses that are admissible for both binary and multi-class classification problems. To demonstrate the usefulness of the proposed framework, we present some multicategory kernel machines and several new multi-class boosting algorithms. keywords: Multi-class classification, Admissible Loss, Margin Vector, Empirical risk minimization, Convexity. ∗Address for correspondence: Hui Zou, 313 Ford Hall, School of Statistics, University of Minnesota, Minneapolis, MN, 55455. Email: [email protected].